Semi-supervised Data Clustering with Coupled Non-negative Matrix Factorization: Sub-category Discovery of Noun Phrases in NELL’s Knowledge Base

The standard non-negative matrix factorization (NMF) is a popular method to obtain low-rank approximation of a non-negative matrix, which is also powerful for clustering and classification in machine learning. In NMF each data sample is represented by a vector of features of the same dimension. In practice, we often have good side information for a subset of data samples. These side information might be binary vectors that indicate human provided class labels, or generic vectors in a different feature space. In this paper we propose the coupled non-negative matrix factorization (CNMF) method to automatically incorporate the side information of a subset of data. In CNMF, the matrix for data samples with or without side information in the original feature space and the matrix for data samples with side information in the new feature space are coupled together and iteratively optimized. Because of different qualities of the side information, a trade-off parameter is introduced to determine the importance of the side information, and we give a cross validation method to choose its value. The time complexity of the CNMF method could be several times bigger than the original NMF method, but still in the same order. As an example of implementing the CNMF method, we look into the knowledge base of the CMU Never-Ending Language Learning (NELL) project and find sub-categories of noun phrases.